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PROJECT  GOALS  AND  OBJECTIVES 

The  PI  and  Co-I  are  examining  mesoscale  predictability  with  the  goal  of  improving  the  utility  of 
ensemble  forecasts  (EFs)  at  ranges  of  12  hours  to  2  days.  Our  research  addresses  the  issues  of  initial 
condition  uncertainty  (ICU)  for  mesoscale  analyses  and  the  merit  of  calibration  of  output  from 
ensemble  prediction  systems  (EPSs)  by  artificial  neural  networks.  The  PI  also  serves  as  Chief  Scientist 
to  Dr.  Scott  Sandgathe  for  ONR  initiative  on  Predictability  in  the  Atmosphere  and  Ocean. 

DOCUMENTATION  OF  ANALYSIS  UNCERTAINTY 

Despite  years  of  ongoing  research,  there  is  still  spirited  debate  on  how  to  generate  initial  perturbations 
for  medium-range  EPS’s  with  global  models.  Research  on  perturbation  design  for  mesoscale  limited- 
area  models  is,  at  best,  in  its  infancy.  It  is  clear  however,  that  whatever  strategy  is  employed  (dynamic 
or  statistical),  initial  perturbations  must  be  properly  constrained  by  our  best  estimate  of  analysis 
uncertainty. 

Several  approaches  come  readily  to  mind.  The  ideal  approach  would  be  the  successful  integration  of 
the  data  assimilation  system  (DAS)  and  the  EPS,  a  so-called  extended  Kalman  filter  (Houtekamer  and 
Mitchell  1998).  Another  approach  would  be  Observing  System  Simulation  Experiments  (OSSEs)  with 
a  four-dimensional  operational  DAS,  which  is  perhaps  the  best,  proven  way  to  address  the  problem.  A 
third  method  would  be  a  thorough  documentation  of  analysis-analysis  differences  from  different 
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Figure  1:  Standard  deviation  of  differences  between  the  ETA  and  NGM  initial  analyses  during  the 
period  1200  UTC  24  August  1998  to  1200  UTC  30  August  1998.  Precipitable  water  (top,  Cl:  0.1”); 
900  mb  vector  wind  ( bottom ,  Cl:  5  knots).  Color  fill  denotes  the  ratio  of  variance  of  analysis 
differences  to  the  variance  of  the  ETA  analysis:  values  <0.5  (green),  values  between  0.5  and  1.0 
(yellow),  values  >1.0  (red).  Black  shading  masks  areas  where  model  fields  are  below  the  ground. 
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analysis-forecast  systems.  This  approach  defines  a  “component"  of  the  analysis  uncertainty.  Although 
this  methodology  is  not  as  insightful  as  an  integration  of  the  DAS  and  EPS  or  a  comprehensive  OSSE, 
it  is  currently  tractable,  very  economical,  and  useful  guidance  can  be  quickly  obtained. 

The  PI  is  comparing  differences  between  NCEP  LAM  analyses  from  the  NGM  (Hoke  et  al.  1989)  and 
ETA  (Rodgers  et  al.  1995)  model.  It  is  important  to  consider  LAM  fields  since  scales  not  resolved  by 
global  analyses  are  presumably  analyzed  with  greater  certainty  over  the  data  rich,  North  American 
continent.  Figure  1  shows  the  standard  deviation  of  differences  between  the  ETA  and  NGM  analyses 
for  the  period  24-30  August  1998.  Plots  are  shown  for  the  precipitable  water,  900  mb  temperature,  and 
900  mb  vector  wind.  The  dominant  synoptic  features  during  the  period  were  hurricanes  Bonnie  along 
the  East  Coast,  Danielle  over  the  Bahamas,  and  Howard  in  the  eastern  Pacific.  Note  that  these  regions 
are  characterized  by  large  absolute  uncertainty,  consistent  with  the  conventional  notion  that  analyses 
are  more  uncertain  near  regions  with  growing  disturbances.  In  fact,  the  normalized  variances  exceed 
unity  over  the  tropical  oceans,  which  indicates  the  information  content  of  tropical  analyses  for  the 
period  is  questionable  at  best. 

NEURAL  NETWORK  POST  PROCESSING  OF  ENSEMBLE  FORECAST  PRODUCTS 

Because  forecast  fields  produced  by  any  NWP  model  always  contain  errors  due  to  model  deficiencies 
(e.g.  lack  of  resolution,  inadequate  parameterizations,  truncation  error,  etc),  raw  model  output  is  often 
statistically  post-processed  to  mitigate  their  impact.  Post-processing  also  provides  a  way  to  relate 
model  output  fields  to  weather  elements  not  explicitly  forecast  by  the  NWP  model  (e.g.  visibility, 
probability  of  thunder). 

There  are  many  viable  ways  to  generate  statistical  forecasts  and  calibrate  NWP  model  (e.g.  Marzban 
and  Stumpf  1998).  The  technique  currently  in  use  at  NCEP  (e.g.  Carter  et  al.  1989)  is  Model  Output 
Statistics  (MOS).  MOS  is  based  on  multiple  linear  regression  (MLR)  and  typically  reduces  the  error 
variance  by  20%.  To  yield  stable  statistics,  MOS  requires  a  data  set  of  several  seasons  for  training  that 
is  averaged  over  a  broad  geographic  region,  and  it  typically  involves  the  screening  of  a  couple  hundred 
predictors.  The  need  for  such  long  training  data  sets  is  a  major  shortcoming  MLR. 

The  calibration  of  EPS  output  presents  even  greater  challenges  than  deterministic  forecasts  because  of 
increased  dimensionality  of  the  output.  Recent  results  indicate  that  strategies  besides  MLR  can  be 
employed  for  EPS  output  with  equal  success  (e.g.  Hamill  and  Colucci  1998),  methods  that  correct  for 
biases  and  under  or  over  dispersion  of  ensemble  spread  and  require  much  shorter  training  periods  than 
MOS. 

The  mature  fields  of  digit  signal  processing  and  artificial  intelligence  offer  other  possible  paradigms  to 
explore.  Consider  artificial  neural  networks  (ANNs),  computer  algorithms  designed  for  nonlinear 
optimization.  Impressive  applications  of  ANNs  to  remote  sensing,  atmospheric  diagnosis  and  weather 
prediction  are  beginning  to  appear.  A  plethora  of  applications,  mostly  diagnostic  or  prognostic  based 
on  extrapolation  from  observations  and  analyses,  recently  appeared  in  the  preprint  volume  of  AMS 
First  Conference  on  Artificial  Intelligence  (1998);  an  excellent  overview  of  NN  applications  to 
operational  forecasting  is  given  by  Christopherson  (1998)  in  the  said  volume. 

The  calibration  NWP  output  by  NNs,  though  limited,  are  just  as  impressive  (Eckert  et  al  1996,  Hall  et 
al.  1999).  The  Pi’s  (Mullen  et  al.  1999)  extended  ANN  processing  to  QPF  output  from  the  pilot 
ETA/RSM  ensemble  data  set  that  was  also  used  by  Hamill  and  Colucci  (1998).  We  found  that 
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calibration  of  12-24  h  rainfall  totals  at  stations  with  a  back  propagating  ANN,  trained  with  just  12 
independent  case  days,  was  competitive  with  MOS.  Results  (Mullen  et  al.  1999,  in  preparation) 
obtained  during  this  summer  under  ONR  support  are  shown  in  Fig.  2.  Note  that  the  0.10”  threshold 
exhibits  close  to  perfect  reliability.  Even  the  0.25”  threshold  (not  shown),  trained  by  a  sample  with 
only  240  episodes  of  0.25”  or  greater  out  of  3900  total  reports,  shows  improvement  over  MOS,  with 
nearly  perfect  reliability  for  probabilities  of  70%  or  lower.  The  refinement  portion  of  the  diagram  (not 
shown)  shows  that  the  0.01”  and  0.10”  thresholds  are  sharp,  but  the  0.25”  threshold  with  no  100% 
forecasts  is  not  as  sharp. 
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Figure  2.  Reliability  diagrams  for  the  0.10”  threshold.  Shown  are  results  for  the  RAW 
ensemble  (blue),  NET  processing  of  the  ensemble  (red),  MOS  (green),  and  NET 

processing  of  MOS  (yellow). 


Under  ONR  support  in  FY99,  the  PI  and  Co-I  will  further  examine  the  robustness  of  ANN  processing 
of  EPS  output.  We  will  1)  screen  predictors  and  predictands  other  than  precipitation,  2)  train  in  one 
geographic  region  and  test  in  another  and  3)  processing  two  modeling  systems,  a  global  model  and  a 
mesoscale  LAM.  We  anticipate  that  NN  processing  of  the  global  and  mesoscale  LAM  EPS  output  will 
yield  improvements  in  accuracy  comparable  to  those  reported  by  Hall  et  al.  (1999). 
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